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Abstract — This paper studies a stochastic game theoretic 
approach to security and intrusion detection in communication 
and computer networks. Specifically, an Attacker and a Defender 
take part in a two-player game over a network of nodes 
whose security assets and vulnerabilities are correlated. Such a 
network can be modeled using weighted directed graphs with 
the edges representing the influence among the nodes. The game 
can be formulated as a non-cooperative zero-sum or nonzero- 
sum stochastic game. However, due to correlation among the 
nodes, if some nodes are compromised, the effective security 
assets and vulnerabilities of the remaining ones will not stay 
the same in general, which leads to complex system dynamics. 
We examine existence, uniqueness, and structure of the solution 
and also provide numerical examples to illustrate our model. 

I. INTRODUCTION 

Today, as computer networks become ubiquitous, network 
security and intrusion detection (ID) play a more and more 
important role. The main task of an intrusion detection sys- 
tem (IDS) is to detect intrusions and report them to a system 
administrator. Among various approaches, non-cooperative 
game theory has recently been employed extensively to study 
ID problems [l]-[6]. 

In a general setting, a security game is defined between 
two players: an Attacker and a Defender (the IDS). A 
formulation of security games as static games can be found 
in [1]. In [3], the authors consider security games with 
imperfect observations and use the finite-state Markov chain 
framework to analyze such games. The work in [4] employs 
the framework of Bayesian games to address the intrusion 
detection problem in wireless ad hoc networks, where a 
mobile node viewed as a player confronts an opponent whose 
type is unknown. 

In [5], the author examines the intrusion detection problem 
in heterogenous networks as a nonzero-sum static game. In 
a complex network, nodes are of different levels of impor- 
tance to the Defender, and also appear variably attractive 
to the Attacker. Heterogeneity also stems from hierarchy 
and correlation among nodes. It is thus essential to consider 
scenarios where nodes have different security assets. Also, 
apart from a node's security asset, if we take into account 
the players' motivations, the cost of attacking, the cost of 
monitoring, and other factors, the game is no longer a zero- 
sum one. Using the Nash Equilibrium (NE) solution concept, 
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the analysis allows one to compute the Attacker's optimal 
strategy as a probability mass distribution on the nodes 
to attack. Similarly, the Defender's optimal strategy is a 
probability mass distribution on the nodes to monitor (to 
collect and process data and detect attacks). However, in 
this work [5], the security assets are still assumed to be 
independent. Also, the dynamics of the ID problem when 
nodes are compromised along the play have not been taken 
into account. 

The work in [6] addresses this problem using the frame- 
work of zero-sum stochastic games [8]. The network is now 
modeled as a discrete-time or continuous-time Markov chain 
where the network states are defined by the states (com- 
promised or not) of the constituent nodes. This formulation 
thus takes into account the dynamics of the problem and 
allows one to incorporate correlation among nodes in terms 
of vulnerability. The analysis is nonetheless limited to zero- 
sum games and again, the security assets are considered to 
be independent. 

This paper attempts to extend these earlier works to con- 
struct a more comprehensive network security and intrusion 
detection model. We develop a network model based on 
linear influence networks proposed in [7]. This model, when 
used under the framework of stochastic games, permits us to 
take into consideration the correlation among the nodes in 
terms of both security assets and vulnerabilities. 

The rest of this paper is organized as follows. In the 
remaining part of this section, we summarize the notations 
and variables used throughout this paper. Next, in Section 
Ull we introduce two linear influence network models for 
security assets and vulnerabilities. In Section [Till we formu- 
late the security game based on these models as a zero-sum 
stochastic game and present results on existence, uniqueness, 
and structure of the solution. We then provide a numerical 
example in Section [IV] Finally, some concluding remarks of 
Section [V] end the paper. 

Summary of notations and variables used in this paper 

• jV: Set of nodes in the network. 

• n: Number of nodes in the network. 

• S s : Set of edges representing the influence among node 
security assets. 

• S v : Set of edges representing the influence among node 
vulnerabilities. 

• eif. A directed edge from node ; to node j, e (/ € <S S or 
eij € S v . 

• ^s- Weighted directed graph for node security assets, 



8v: Weighted directed graph for node vulnerabilities, 

% = {JV,£ V } 

I, Uf. Influence matrix for security assets and its entries. 
Wif. Influence of node i on node j in terms of security 
assets, where i,j e JV 

s= {s\,S2, ■ ■ ■ ,s„}: Vector of independent security as- 
sets. 

x = {xi,X2, ■ ■ ■ ,x n }: Vector of effective security assets. 
H, hjf Support matrix and its entries, signifies the 
support that node ; gives node j (against attacks), < 
hij < 1 V;',;' G JY. 

hf Support to node j, j G JY, hj = YH=\ hij- 

pi,: Probability that node j is compromised when player 

1 (the Attacker) attacks, player 2 (the Defender) does 
not defend the node, and the support to node j is equal 
to 1 (full support). 

p ] n0 : Probability that node j is compromised when the 
Attacker attacks, the Defender does not defend the node, 
and the support to node j is equal to (no support). 
p J dl : Probability that node j is compromised when the 
Attacker attacks, the Defender defends the node, and 
the support to node j is equal to 1 (full support). 
p J dQ : Probability that node j is compromised when the 
Attacker attacks, the Defender defends the node, and 
the support to node j is equal to (no support). 

S\,S2, ■ ■ -5 P } : States in the state space of the system. 

ri , 1^2 , . . . T p } : Game elements of the stochastic game, 
each of which corresponds to a state of the system. 
p k r : Probability that the network goes back to state Si, 
given that it is currently in state S^, the Attacker attacks 
one node and the attack fails. 

p k : Probability that the game ends given that it is 
currently in state Sk, the Attacker attacks one node and 
the attack fails. 

PQ r : Probability that the network goes back to state Si, 
given that it is currently in state and the Attacker 
does not attack any node. 

pi e : Probability that the game ends given that it is 
currently in state and the Attacker does not attack 
any node. 

a k :\ Instant amount that player 2 pays player 1 at game 
element Tj, if player 1 plays pure strategy i and player 

2 plays pure strategy j. 

qfj\ Probability that both players have to play game 
element T/ next, given that they are currently at game 
element I^, if player 1 plays pure strategy i and player 
2 plays pure strategy j. 

cm: Probability that the game ends given that they are 
currently at game element Y^, if player 1 plays pure 
strategy i and player 2 plays pure strategy j. 
m^. Number of pure strategies for player 1 at game 
element Y^. 

n^. Number of pure strategies for player 2 at game 
element F^. 

p (p = 2"): Number of game elements of the stochastic 
game, or the number of states of the state space. 



• afy A collective entry that includes the instant payoff 
and the transition probabilities to all game elements, 
(Xjj = a*- + Y$=i Hij^u given that the players are cur- 
rently at game element Y^, player 1 plays pure strategy 
i, and player 2 plays pure strategy j. 

• b\f Value of cc k j when we replace game elements T/'s 
with their values, b k j = a k j + Lf =1 <?f/V/. 

• yf: Probability that player 1 plays pure strategy i when 
playing game element Yk at the f-th stage of the game. 
For stationary strategies [8], the superscript t will be 
omitted. 

• z k j: Probability that player 2 plays pure strategy j when 
playing game element at the f-th stage of the game. 

• 3> fa , (k = 1, ... ,p, t = 1,2, ...): Strategy for player 1, a 
set of m^-vectors each of which is a mixed strategy of 
player 1 at game element Y^ and f-th stage of the game. 

• z kt , (k — 1, ... ,p, t — 1,2, ...): Strategy for player 2, a 
set of rc^-vectors each of which is a mixed strategy of 
player 2 at game element Y^ and f-th stage of the game. 

• c\: Pure strategy i for the Attacker at game element IV 

• d k : Pure strategy j for the Defender at game element 

r*. 

• Psirf'dj): Probability that the attack is successful given 
that the Attacker plays pure strategy c\ and the Defender 
plays pure strategy d k at game element IV 

• v = ( v i i v 2, • • • i Vp)'- Value vector of the stochastic game. 

• val(B): Value of the zero-sum matrix game given by the 
matrix B. 

II. LINEAR INFLUENCE NETWORK MODELS FOR 
SECURITY ASSETS AND FOR VULNERABILITIES 

We present in this section a network model based on the 
concept of linear influence networks [7]. The network will be 
represented by two weighted directed graphs, one signifying 
the relationship of security assets and the other denoting 
vulnerability correlation among the nodes. 

A. Linear influence network model for security assets 

For a particular node, the general term security asset is 
used to signify how important the node is to the network. 
All the security assets of a network can be modeled as a 
weighted directed graph @ s = \ jY ,§s\ where jY is the set 
of nodes, and the elements of set S s represent the influence 
among the nodes. Let n be the cardinality of ,jV . For each 
edge etj G S s , we denote an associated scalar wn that signifies 
the influence of node i on node j, where i, j G jV . The entries 
of the influence matrix I are then given as follows: 

/.. = / *"y if *ye** m 
lJ \ otherwise, 

where < w, 7 < 1 V/,y G JY and Yll=\ wy = 1, V/ G JV. 
Note that here we allow for the edges of the form wjj = 
1 — Y!}=i i^j w 'j' which signifies the portion of influence of a 
node on the independent security asset of itself. 

Let s = {si,S2, ■ ■ ■ be the vector of independent secu- 
rity assets. The vector of effective security assets, denoted 



by x = {xi,X2, ■ ■ ■ ,x n } can then be computed by the influence 
equation: 

x = Is. (2) 



With the condition £? = 



l.Vy =e Jf , we have that 



L = EE W = E E w H s i 

i=l i=l;'=l ;=1/=1 

7=1 i=l j=l 



(3) 



Therefore, the sum of all the effective security assets is 
equal to the sum of all the independent security assets. 
The influence matrix thus signifies the redistribution of 
security assets. The independent security asset of a node ; 
is redistributed to all the nodes in the network that have 
influence on i (including itself). When a node is down, the 
node itself and all the edges connected to it will be removed 
from the graph. Thus the security loss of the network will be 
the node's effective security asset (instead of its independent 
security asset). Conversely, if a node is brought back to the 
network, it regains its original influence on other nodes. In 
either case, the entries of the influence matrix have to be 
normalized to satisfy £" =1 w ij = lj Vj £ For a quick 
justification of this linear influence model, consider a GSM 
network, where a base station controller (BSC) i controls 
several base transceiver stations (BTS), including BTS j. If 
a BSC fails, all the BTSs connected to it will be out of 
service. On the contrary, if only one BTS is compromised, 
the communication among the subscribers under other BTSs 
should not be affected (provided that the rest of the network 
is up and running). In such a situation, we can have for 
example, wjj = 0.7 and wn = 0.3. If the BSC is down, there 
is still an amount of security asset O-lsj left, even though 
the BTS is not in service anymore. The reason is that, if this 
BTS gets connected to another BSC (or if the original BSC 
is up again), they will together create an added security asset 
for the network. We present in what follows an example to 
illustrate the linear influence network model. 
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Fig. 1. A linear influence network for security assets of a three-node 
network. 

Example 1: Suppose that we have a network of three 
nodes with correlations as shown in Fig. Q] As shown in Fig. 
12 the states of the system are given as {S\ , S2, ■ ■ ■ S p } (p = 



2") where E {0, 1}", k = l,...,p. Here a node is said 
to be in state 1 if it is compromised and otherwise. Note 
that we consider a discrete-time Markov chain where the 
system can transit from one state to any state of the state 
space (including the original state). The influence equation 




Fig. 2. An example state diagram for the network in Fig. [T] 





Fig. 3. Changes in a linear influence network for security assets when 
nodes are compromised (Example [TJ. 
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Now suppose that node 1 is compromised; then the in- 
dependent security asset of node 3 will remain the same, 
The independent security asset of node 2 will 



.(2) _ .(1) 
s 3 — s 3 



be decreased by an amount corresponding to the influence 

(1* 



of node 1 on node 2: sf> = -0.24 = °- 8 4 ■ Also ' 



the influences on each node have to be normalized to have 
l^-Wji = 1. Thus we now have W32 = 1/8 and W22 = 7/8, and 
the influence equation becomes 



(5) 




Thus we can see 



(2) 
K 2 



= (7/8)4-0.74 



s (i) 



= (i/8)4^+4" ; = o.i4 1 



After node 1 goes down, the effective security asset of 
node 2 remains the same, while that of node 3 is decreased 
by an amount representing its influence on node 1. 



Now if node 3 is in turn compromised, we have a network 
with one node as in Fig. [3] We have 
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/8 = (7/8)4 



0.7s 



(l) 



B. Linear influence network model for vulnerabilities 

In this subsection, we use the linear influence network 
model to represent the correlation of node vulnerabilities in 
a network. Beside the correlation of security assets, nodes 
also have influence on others' vulnerabilities. For example, 
within a corporate network, if a workstation is compromised, 
the data stored in this computer can be exploited in attacks 
against other workstations; these latter computers thus will 
become more vulnerable to intrusion. Under the framework 
of stochastic games, this kind of influence is readily incor- 
porated. For instance, in the network of Example Q] if the 
Attacker attacks node 1, and the Defender decides not to 
defend this node, the probability that the system goes from 
(0,1,0) to (1,1,0) will be greater that the probability that 
the system goes from (0,0,0) to (1,0,0), if node 2 has some 
influence on node 1 in terms of vulnerability. For ey 6 $ v , 
we define the support matrix as follows 



H 



hij if ey G S v 
otherwise, 



where hn signifies the support that node i gives node j 
(against attacks), < ha < 1 Vz, j G JV. The support to node 
j, j G jV is defined as 



[■=] 



where < hj < 1, Vj G JV . Unlike the model for security 
assets, here we do not normalize hj. When a node that 
supports node j is down, hj will decrease, and thus the 
probability that node j is compromised under attack will 
increase. Let us denote by p{ the probability that node j is 
compromised at each state. We assume an affine relationship 
between p{ and hj as follows: 

• If node j is not attacked then pi = 0. 

« If node j is attacked, and the Defender is not defending 
this node, pi = p J n0 - (p J n0 - p ] nl )hj, where p J nl and p J n0 
are the probabilities that the node is compromised given 
that the support is equal to 1 (full support) and (no 
support), respectively (p J nl < p J n0 ). 

• If node j is attacked, and the Defender is defending this 
node, pi = p J d0 - {p J d0 -p J in )hj, where p J dl and p J d0 are 
the probabilities that the node is compromised given that 
the support is equal to 1 and 0, respectively (p J dl < p J d0 ). 

• Also, it is assumed that p J dl < p J nl and p ] d0 < p J nQ . 

A weighted directed graph for network vulnerabilities is 
shown in Fig. |4] 





Fig. 4. A linear influence network for vulnerabilities and the changes of 
supports when one node is compromised. 



III. THE NETWORK SECURITY PROBLEM AS A 
ZERO-SUM STOCHASTIC GAME 

A. A brief overview of zero-sum stochastic games 

In this subsection, we provide a brief overview of zero- 
sum stochastic games based on [8]. A stochastic game 
consists of p game elements F^, k = I,..., p. Each game 
element is associated with an m# x n^ matrix, whose entries 
are given by 



•j 



4- 



E4^ 



(6) where q 



kl 

ij 

14 

1=1 

Expression 
element 



> 0, Z = l, z'= l,...,m k , j = l,. 
1, Vk,i,j. 



< 



(8) 



(9) 



dH) can be interpreted as follows. At game 
if player 1 chooses pure strategy z and player 
(7) 2 chooses pure strategy j, player 2 has to pay player 1 an 
amount a\j. Furthermore, there is a probability qfj that both 



players have to play game element F/ next, and a probability 



i=i 



(10) 



that the game will end. With condition (0, the probability 
of infinite play is guaranteed to be zero, and the expected 
payoff of player 1 (or the expected loss of player 2), which 
is accumulated through all the stages of the game, is finite 
[8]. 

A strategy for player 1 is a set of zii^-vectors, denoted by 
A k=l,...,p, t = 1,2,. 



each of which satisfies 



Erf 



yf >o 



(ID 

(12) 



Here yf is the probability that player 1 plays pure strategy 
i if he is playing game element at the f-th stage of the 
game. A strategy is said to be stationary if the vectors y kt 
are independent of t for all k. In this case, the superscript t 
can be omitted. Similarly, a strategy for player 2 is a set of 
zz^-vectors, z kt , where = 1 an d > 0- Given a pair 



of strategies, we can compute the vector of expected payoffs 
v= (\'i,V2,...,v ; ,), where v&, k = l,...,p is the expected 
payoff (to player 1) if the first stage of the game is F^. 

With the above settings, it is known [8], that we can 
replace the game element by the value component 

v k = val(B k ), (13) 

where va^B^) is the value (in mixed strategies) of the matrix 
game B k , and B k is the m k x n# matrix whose entries are given 

by 

b ij= a ij + t^h- ( 14 ) 
1=1 

B. A zero-sum stochastic game model for network security 

In this subsection we formulate the security problem as a 
zero-sum stochastic game. This is a modified version of the 
game presented in [6], applied to the linear influence network 
model proposed in Section [TT] At each state k, k = 1 , . . . ,p, 
the Attacker's pure strategies consist of m k = n + 1 actions, 
where « is the number of nodes in the network: 

• Attack one of n nodes, c\, where i = l,...,n. 

• Do nothing, cjL = 0. 

Note that this strategy space is for use with more general 
payoff formulations. However, with the payoff formulation 
in this paper, the Attacker will not have motivation to attack 
a node that is already compromised, unless all the nodes 
have been compromised. For each k, the Defender's pure 
strategies are {^f}, where 

• Defend node i, df,i = 1 , . . . , «t — 1 , 

• Do nothing, = 0, 

where n k = = n + 1. For each possible combination of the 
Attacker's and the Defender's pure strategies, the entries of 
the payoff matrix are: 

where a\- = p k s (c\ ,dj)^(i), p k (c k ,dj) is the probability that 
the attack is successful, and x k (i) is the effective security 
asset of the node being attacked, i. Note that once a node 
is compromised, the effective security assets and the sup- 
ports of the remaining nodes have to be recalculated as in 
Example [T] and Fig. |4] As mentioned in Subsection III-BI 
the probabilities p k , and thus qf,, are dependent on the 
supports to the nodes, and are therefore affected by the 
correlation in vulnerabilities of the nodes. It can be said 
that once we have incorporated node vulnerabilities into 
our model, we have already implicitly taken care of the 
cost of attacking/defending. For example, if a node is of 
high security asset but difficult to compromise (the transition 
probability to the compromise state is small), the Attacker 
may turn to another node with a smaller security asset, which 
is easier to attack. 

At a state Sj, if the Attacker chooses to attack one node 
and the attack fails, there is a probability p k r S (0,1) that 
the network will go back to state Si (which means the 
Defender has detected the Attacker and managed to restore 



all the compromised nodes and the game restarts at Si), 
and a probability p k e S (0, 1) that the game will end (which 
means the Defender has detected the Attacker and stopped 
him from further intruding). Note that p k + p k < 1 with 
equality only when St : = Si (0,0, . . . ,0). Similarly, at one 
point, if the Attacker chooses not to attack at all, there is 
a probability pL G (0,1) that the network will go back to 
state Si, and a probability p\ e € (0,1) that the game will 
end. Given 0<p J dv p J nV p J d0 , p J n0 < 1, j € Jf , p* p* p\ r , 
and p^ e , k = 1,. . . ,p, and the support matrix H, p k and qfj 
can be calculated using the equations in Subsection III-BI A 
numerical example is shown in Section ITVl 

C. Existence, uniqueness, and structure of the solution 

We present in this subsection some analytical results for 
the game given in IIII-BI based on zero-sum stochastic game 
theory [8], [9]. 

Proposition 1: In the zero-sum stochastic game given in 
IIII-BI the probability of infinite play is zero and the expected 
payoff of the Attacker (which is also the expected cost of 
the Defender) is finite. 

With the setup in llll-BI we can show that qfj = 1 -£f =1 q\) > 
0, Vk and V i,j of each game element F^. Thus the propo- 
sition is proved using the theory of stochastic games. 

Proposition 2: (Theorem V.3.3 [8]) In the zero-sum 
stochastic game given in IIII-BI there exists exactly one vector 
v = (vi, V2, • • • , v p ) that satisfies (fT3l and (Tl4l i. 
Using the results from IIII-Al we can then compute the NE 
of the game, which is a pair of stationary mixed strategies 
for the Attacker and for the Defender at each state. 

Proposition 3: (Theorem V.3.3 [8]) The vector v = 
(vi, V2, . . . , v p ) that satisfies ([T31 and (TPfl i can be derived 
through the following recursive equations: 

v° = (0,0,..., 0), (16) 

b ij = Jj + ttfrf, ( 17 ) 
(=1 

v r k +i = val{B' k )=val{b kr j ). (18) 
We can stop the recursion at a desired level of accuracy and 
then use the current value of vector v = (vi,V2, . . . ,v p ) to 
compute fit using (fl4] i. The mixed strategies of the players 
at each game element Tt are the NE in mixed strategies of 
the matrix game B k . The strategies so obtained will converge 
to optimal stationary strategies of the stochastic game. 

IV. A NUMERICAL EXAMPLE 

In this section, we implement numerical simulation for 
a specific network with three nodes. The setup in IIII-BI 
is carried over with some further assumptions as follows. 
First, we adopt a simplified state diagram as given in Fig. Q] 
Basically, after each time step, we only allow for transitions 
where one more node is compromised, the transition that 
returns to the same state, and the transition back to Si (0,0,0). 
Second, suppose that the influence equation is given as 



follows (Example [U 
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and the support matrix is given by (Fig. HI 



H 




(19) 



(20) 



Finally, p J dl = 0.2, p J nl = 0.4, p> m = 0.5, p J n0 = 0.7, Vj € ^, 



= 0.2, VA:^ 1, =0.7, p* =0.3, Vfc = 1, . . . ,p, 
0.2, Vfc^l,p' r = 0.7, and ^ e =0.3, V£=l,...,p. 
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For example, suppose the system is at 
S\ (0,0,0). The next state could be one in 
{Si (0,0,0), S 2 (0,0,1), S 3 (0,1,0), S 5 (1,0,0)}. 
The Attacker's pure strategies include 1,2,3, and 0, which 
mean to attack node 1, node 2, node 3, and do nothing, 
respectively. Similarly, the Defender's pure strategies include 
1,2,3, and 0. Using the above results, we have that 



a 



,i 
1 1 

ill 

ill 

l l\ l 



(i) 



(i-pi(i,i))(i 
pi (i,i), 

0V/V1.5, 



y e ), 



where p](l,l) = p d0 - (pdo ~ Pdi ) 1 = Pdi, as at this state, 
node 1 still has full support. Also, there is a probability p g e = 
(1 -pl(l,l))p le >0 that the game will end. If the Attacker 
attacks node 1 and the Defender defends node 2, we have 
that 



in 



"12 

ill 
ill 

1l2 



Pl(h2)x\ , 

(i-r!(i,2))(i 

P. 1 (1,2), 
V/V 1,5, 



P l % 



where p](l 



1) = PnO - (p«0 -p«i)l = Pm, again as at this 
state, node 1 still has full support. Also, there is a probability 
p]f = (1 -pj(l,2))p le > that the game will end. Now, 
suppose that the system is at S$ (1,0,0). The next state could 
be one in {Sx (0,0,0), S 5 (1,0,0), S 6 (1,0,1), S 7 (1,1,0)}. 
The Attacker's pure strategies include 2,3, and 0, which 
mean to attack node 2, node 3, and do nothing, respectively. 
Similarly, the Defender's pure strategies include 2,3, and 0. 
Now we have that 



a 22 

41 
41 

122 
5; 
122 



p]{2,2)x 2 



(5) 

R?(2,2), 
(1-^(2,2))^, 
(1-^(2,2))(1- 
VjV 1,5,7, 



it -pi), 



where p 2 s {2,2) = p^ — {p 2 d(j ~ p^j)0.8, as at this state, node 
2 has a support of 0.8. Also, there is a probability p 5e = 
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Node 1 


Node 2 


Node 3 


Do nothing 


1 (0,0,0) 


0.6126 





0.3874 





2 (0,0,1) 


0.3817 


0.6183 








3 (0,1,0) 


0.6415 





0.3585 





4 (0,1,1) 
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5 (1,0,0) 





0.6568 


0.3432 





6 (1,0,1) 





1 








7 (1,1,0) 








1 





8 (1,1,1) 


0.25 


0.25 


0.25 


0.25 



TABLE I 

Optimal strategies for the Attacker at each game element 
(GE). 



GE 


Node 1 


Node 2 


Node 3 


Do nothing 


1 (0,0,0) 


0.0702 





0.9298 





2 (0,0,1) 


0.6614 


0.3386 








3 (0,1,0) 


0.0869 





0.9131 





4 (0,1,1) 


1 











5 (1,0,0) 





0.034 


0.966 





6 (1,0,1) 





1 








7 (1,1,0) 








1 





8 (1,1,1) 


0.25 


0.25 


0.25 


0.25 



TABLE II 

Optimal strategies for the Defender at each game element. 



(1 — Pj(2,2))p 5e > that the game will end. The other entries 
of other game elements can be calculated in a similar way. 
Using the recursive procedure given in Proposition [3] we 
can then compute the optimal strategy of each player and 
the value of the game. The value vector converges to an 
accuracy of 10~ 4 after 56 iterations. The optimal strategies 
of the Attacker and the Defender, and the value vector are 
given in Tables U [TTH and [Til] As can be seen from Table 
12 for example, when all the nodes are up and running, the 
Attacker wants to attack node 1 with probability 0.6126 and 
node 3 with probability 0.3874, while the Defender wants 
to defend node 1 with probability 0.0702 and node 3 with 
probability 0.9298. Recall that the effective security assets of 
nodes 1 , 2, and 3 at this state are 11, 7, and 22, respectively. 
It is worth noting that the mixed strategies for the players 
can also be interpreted as the way to allocate their resources 
in the security game. 



GE 


1 


2 


3 


4 


Payoffs 


19.6078 


15.8301 


17.9557 


12.3392 


GE 


5 


6 


7 


8 


Payoffs 


17.9659 


13.0283 


15.3228 


7.8431 



TABLE III 

The value vector (the expected payoffs of the Attacker, also 
the expected losses of the defender at each game element). 



V. CONCLUSION 

In this paper we have proposed a new network model 
based on linear influence networks to represent the inter- 
dependence of nodes in terms of security assets and vul- 
nerabilities. We took the first step to formulate the security 
game between an Attacker and a Defender over this network 
using the framework of zero-sum stochastic game theory. 
The optimal solution obtained allows one to comprehend the 
behavior of a rational attacker, as well as to provide IDSs 
with guidelines on how to allocate their resources. Moreover, 
modeling networks with linear influence network models 
helps facilitate solving the security games using software 
programs. As mentioned earlier, apart from a node's security 
asset, if we take into account the players' motivations, the 
cost of attacking, the cost of monitoring, and other factors, 
the game is no longer a zero-sum one. This work thus can 
be extended to nonzero-sum stochastic games, where we 
can address more flexible and practical payoff formulations. 
Furthermore, in many real-world scenarios, neither the At- 
tacker nor the Defender has full knowledge of the network's 
nodes and their correlation. Thus studying stochastic security 
games with incomplete information is an intriguing research 
direction. 
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